A unioning file system combines the namespaces of two or more file systems together to produce a single merged namespace. This is useful for things like a live CD/DVD: one can union-mount a small, writeable file system on top of a read-only DVD file system and have a usable system without needing to transfer the system from the DVD to the root file system. Another use is to export single read-only base file systems via a network file system (NFS) protocol to multiple clients, each with their own small writeable overlay file system union mounted on top.
Union file systems are inherently difficult to implement for a variety of reasons. However, much of the complexity comes from solving the following problems: whiteouts, readdir( ) support, stable inode numbers, and concurrent modifications to more than one branch at a time. For instance, with respect to readdir( ) support, readdir( ) is an operation that occurs when listing files in a directory, such as when you open a folder in the directory. Generally, the main problem with readdir( ) in operation is that it is unknown beforehand how many directory entries there are in the directory. As a result, only a few entries in the directory are read in at a time in order to conserve memory. A “bookmark” is utilized to mark the location where the readdir( ) operation paused and needs to pick back up again on the next read. Typically, this “bookmark” is an identifier of the location in the directory the readdir( ) operation stopped and is specific to each file system. In the realm of unioning file systems, where namespaces are merged, an issue arises with readdir( ) because there are two directories with entries that are listed for one unioned file system, but only one “bookmark” is returned. There is no standard for the “bookmark” between file systems, so the bookmark loses its relevance.
One example of a problem with some current implementations of readdir( ) is that they occur in the kernel. The “bookmark” field is set to the offset within the current underlying directory, minus the sizes of the previous directories. Directory entries from directories underneath the top layer must be checked against previous entries for duplicates or whiteouts. As currently implemented, each readdir( ) system call reads all of the previous directory entries into an in-kernel cache, then compares each entry to be returned with those already in the cache before copying it to the user buffer. The end result is that readdir( ) is complex, slow, and potentially allocates a great deal of kernel memory. Implementations that use application memory rather than kernel memory to cache directory entries have similar unacceptable costs in memory and processing time
This is just one example of the complexities resulting from unioning file systems, and other problems also arise such as optimizing whiteouts, providing stable inode numbers, and dealing with concurrent modifications. As such, a solution to reduce the complexity of union file systems by streamlining the problems with such systems would be beneficial.